Hybrid storage systems (HSS) integrate multiple storage devices with diverse characteristics to deliver high performance and capacity at low cost. The performance of an HSS highly depends on the effectiveness of two key policies: (1) the data-placement policy, which determines the best-fit storage device for incoming data, and (2) the data-migration policy, which dynamically rearranges stored data (i.e., prefetches hot data and evicts cold data) across the devices to sustain high HSS performance. Prior works optimize either data placement or data migration in isolation, which leads to suboptimal HSS performance. Unfortunately, no prior work tries to optimize both policies together. Our goal is to design a holistic data-management technique that optimizes both data-placement and data-migration policies to fully exploit the potential of an HSS, and thus significantly improve system performance. We propose Harmonia, a multi-agent reinforcement learning (RL)-based data-management technique that employs two lightweight autonomous RL agents, a data-placement agent and a data-migration agent, that adapt their policies for the current workload and HSS configuration while coordinating with each other to improve overall HSS performance. We evaluate Harmonia on real HSS configurations with up to four heterogeneous storage devices and seventeen data-intensive workloads. On performance-optimized (cost-optimized) HSS with two storage devices, Harmonia outperforms the best-performing prior approach by 49.5% (31.7%) on average. On an HSS with three (four) devices, Harmonia outperforms the best-performing prior work by 37.0% (42.0%) on average. Harmonia's performance benefits come with low latency (240ns for inference) and storage overheads (206 KiB in DRAM for both RL agents combined). We will open-source Harmonia's implementation to aid future research on HSS.

We aim to better understand the tradeoffs between traditional and reinforcement learning (RL) approaches for energy storage management. More specifically, we wish to better understand the performance loss incurred when using a generative RL policy instead of using a traditional approach to find optimal control policies for specific instances. Our comparison is based on a simplified micro-grid model, that includes a load component, a photovoltaic source, and a storage device. Based on this model, we examine three use cases of increasing complexity: ideal storage with convex cost functions, lossy storage devices, and lossy storage devices with convex transmission losses. With the aim of promoting the principled use RL based methods in this challenging and important domain, we provide a detailed formulation of each use case and a detailed description of the optimization challenges. We then compare the performance of traditional and RL methods, discuss settings in which it is beneficial to use each method, and suggest avenues for future investigation.

Energy storage devices, such as batteries, thermal energy storages, and hydrogen systems, can help mitigate climate change by ensuring a more stable and sustainable power supply. To maximize the effectiveness of such energy storage, determining the appropriate charging and discharging amounts for each time period is crucial. Reinforcement learning is preferred over traditional optimization for the control of energy storage due to its ability to adapt to dynamic and complex environments. However, the continuous nature of charging and discharging levels in energy storage poses limitations for discrete reinforcement learning, and time-varying feasible charge-discharge range based on state of charge (SoC) variability also limits the conventional continuous reinforcement learning. In this paper, we propose a continuous reinforcement learning approach that takes into account the time-varying feasible charge-discharge range. An additional objective function was introduced for learning the feasible action range for each time period, supplementing the objectives of training the actor for policy learning and the critic for value learning. This actively promotes the utilization of energy storage by preventing them from getting stuck in suboptimal states, such as continuous full charging or discharging. This is achieved through the enforcement of the charging and discharging levels into the feasible action range. The experimental results demonstrated that the proposed method further maximized the effectiveness of energy storage by actively enhancing its utilization.

Hybrid storage systems (HSS) use multiple different storage devices to provide high and scalable storage capacity at high performance. Recent research proposes various techniques that aim to accurately identify performance-critical data to place it in a "best-fit" storage device. Unfortunately, most of these techniques are rigid, which (1) limits their adaptivity to perform well for a wide range of workloads and storage device configurations, and (2) makes it difficult for designers to extend these techniques to different storage system configurations (e.g., with a different number or different types of storage devices) than the configuration they are designed for. We introduce Sibyl, the first technique that uses reinforcement learning for data placement in hybrid storage systems. Sibyl observes different features of the running workload as well as the storage devices to make system-aware data placement decisions. For every decision it makes, Sibyl receives a reward from the system that it uses to evaluate the long-term performance impact of its decision and continuously optimizes its data placement policy online. We implement Sibyl on real systems with various HSS configurations. Our results show that Sibyl provides 21.6%/19.9% performance improvement in a performance-oriented/cost-oriented HSS configuration compared to the best previous data placement technique. Our evaluation using an HSS configuration with three different storage devices shows that Sibyl outperforms the state-of-the-art data placement policy by 23.9%-48.2%, while significantly reducing the system architect's burden in designing a data placement mechanism that can simultaneously incorporate three storage devices. We show that Sibyl achieves 80% of the performance of an oracle policy that has complete knowledge of future access patterns while incurring a very modest storage overhead of only 124.4 KiB.

As future energy systems become more decentralised due to the integration of renewable energy resources and storage technologies, several autonomous energy management and peer-to-peer trading mechanisms have been recently proposed for the operation of energy hub networks based on optimization and game theory. However, most of these strategies have been tested either only in simulated environments or small prosumer units as opposed to larger energy hubs. This simulation reality gap has hindered large-scale implementation and practical application of these method. In this paper, we aim to experimentally validate the performance of a novel multi-horizon distributed model predictive controller for an energy hub network by implementing the controller on a complete network of hubs comprising of a real energy hub inter-faced with multiple virtual hubs. The experiments are done using two different network topologies and the controller shows promising results in both setups.

It's official: The next-generation Thunderbolt spec will be called Thunderbolt 5, debuting next year with enough charging power and bandwidth to support eGPUs and a new class of "external AI devices." Intel teased the new Thunderbolt specification at the end of 2022, promising that the next-gen Thunderbolt would continue the trend of doubling bandwidth, all the way to 80Gbps in one direction. That included a promise, now confirmed, that the four lanes of Thunderbolt could be reconfigured to allow three lanes from a laptop to a monitor, rather than two. That will allow the option of a 120Gbps connection to a display, which Intel now refers to as Bandwidth Boost. Thunderbolt 5 will eventually be integrated within Intel's Core platforms, primarily laptops.

There are many benefits of GPUs in scaling AI, ranging from faster model training to GPU-accelerated fraud detection. While planning AI models and deployed apps, scalability challenges--especially performance and storage--must be accounted for. Of these elements, data storage is often the most neglected during the planning process. Because data storage needs, over time, are not always considered while creating and deploying an AI solution. Most requirements for an AI deployment are quickly confirmed through a POC or test environment.

Cloud technology and AI are rapidly changing the state of our technological landscape. Many old forms of technology have started to become obsolete, as a growing number of new tools utilizing these new forms of technology are making things easier. However, the cloud and big data are also offering some benefits that help older forms of technology stay relevant. USB drives are an example. Despite the growing relevance of cloud technology, global customers still spend over $35 billion on USB devices and the market is growing over 9% a year.

SNIA held its Persistent Memory and Computational Storage Summit, virtual this year, like last year. Let's explore some of the insights from that virtual conference from the first day. Dr. Yang Seok, VP of the Memory Solutions Lab at Samsung spoke about the company's SmartSSD. He argued that computational storage devices, which off-load processing from CPUs, may reduce energy consumption and thus provide a green computing alternative. He pointed out that data center energy usage has stayed flat at about 1% since 2010 (in 2020 its was 200-250 TWh per year) due to technology innovations.

Google has hired former Intel executive Uri Frank to lead its custom chip division. Apart from Google, many companies have taken to chipmaking in the last few years to build competitive moats. The Intel veteran will serve as the Vice-President of Engineering for server chip design at Google. Uri Frank has over two decades of experience in custom CPU design and delivery experience. His expertise in design engineering at Intel will come in handy for Google.